Prosthetic Devices as Goal-Seeking Agents

In this article we develop the perspective that assistive devices, and specifically artificial arms and hands, may be beneficially viewed as goal-seeking agents. We further suggest that taking this perspective enables more powerful interactions between human users and next generation prosthetic devices, especially when the sensorimotor space of the prosthetic technology greatly exceeds the conventional myoelectric control and communication channels available to a prosthetic user. As a principal contribution, we propose a schema for thinking about the capacity of a human-machine collaboration as a function of both the human and machine’s degrees of agency. Using this schema, we present a brief analysis of three examples from the literature where agency or goal-seeking behaviour by a prosthesis has enabled a progression of fruitful, task-directed interactions between a prosthetic assistant and a human director. While preliminary, the agent-based viewpoint developed in this article extends current thinking on how best to support the natural, functional use of increasingly complex prosthetic enhancements. I. THE PROSTHETIC FUTURE Upper-limb prosthetic devices have evolved over the last several hundred years from crude iron hands to exquisitely designed bionic body parts [1], [2]. However, despite great improvements in quality of life for those with lost limbs, the state-of-the-art has yet to create a satisfactory substitute for the nearly 1 in 200 Americans living with amputations [1]–[3]. Significant advances have been made. Extensions to hardware, software, and interfaces have paved the way for increasingly more adaptive and functional prosthetic technologies. Of note, the actuation capabilities of both commercial and experimental powered upper-limb prostheses far surpass the ability of users to manipulate all available degrees of control [4]. Advances in software, shared control between the human and the prosthesis, and machine learning in the device itself are now needed to fully bridge the gap between a user and their prosthesis [2], [5]. Prostheses are interesting in part because of the intimate way control is shared between a human and their device (Fig. 1); from a technical standpoint, the prosthetic setting is both challenging and appealing due to the dynamic, nonstationary nature of human environments [8]. Prosthetic devices must therefore maintain and update a representation of their environment, sharing some subset of their perception of the world with their human user. Modern technology enables increasingly powerful shared representations. Muscular, neural and osseo-integration allow for direct connections between the human and the device [1], [2], [9]. Onboard cameras have been shown to facilitate real-time visual Fig. 1. Examples of human-prosthesis interaction. Left: a subject with an amputation using the University of Alberta Bento Arm [6] with conventional myoelectric control to complete a manipulation task. Right: control of a supernumerary limb by a non-amputee subject [7]. object tracking and object recognition for grasp pre-shaping [10]. Microphones and speakers facilitate natural-language interactions with devices, as seen in related domains [11], [12]. Sensory feedback and surgical practice have further evolved to restore sensation to prosthesis users [13], [14]. Future prosthetic devices will receive an unprecedented density of data about the user, their needs, and their environment. This stream of data will need to be skillfully leveraged to enable the coordination of vast numbers of complementary actuators and functions. As such, and in addition to advances in communication streams between the device and the human, prosthetic limbs will soon need to actively build and improve their representation of the world around them. Prosthetic limbs will be required to structure a vast amount of data to better make decisions in support of their users’ needs and goals. The principal contribution of this work is therefore to suggest that a prosthetic device should be an agent—i.e., that it should be an autonomous system that both has and seeks goals. In more general terms, we propose that the parts of a larger information processing system (e.g., both sides of a tightly coupled human-machine interface) are well thought of as each being full information-processing systems with goals. We further suggest that for maximum benefit all parts of an interface should model the other parts as being goalseeking systems. In the remainder of this manuscript we will develop the intuition behind an agent-based viewpoint. II. GOAL-SEEKING COMMUNICATION AND CONTROL There are multiple means by which the human and an agent—e.g., an assistive robot like a prosthesis—can beneficially interact to achieve the human’s objectives [11], [15], [16]. In much of the existing literature, one or more feedback channels are used as a means by which a non-expert can train, teach, and interact with a system without explicitly programming it. This shaping allows for the human to learn how the robot accepts and interprets feedback, and for the robot to learn what the human’s goals are for their shared interaction [17]. A selection of representative examples are briefly surveyed below. Pilarski et al. introduced the use of actor-critic reinforcement learning for myoelectric limb control and showed that a user could train a virtual robotic appendage with a single, scalar reward signal provided by the user [18]. Knox and Stone explored a wide variety of strategies for incorporating feedback with environmental reward. They found that Action Biasing and Control Sharing, both using feedback as policy modifiers rather than changing the reward function, produced the best results [19]. Griffith et al. built on the work of Knox and Stone with Advise, a framework to maximize the information gained from human feedback by associating policy labels [20]. Advise outperformed other modern methods in robustness to noise. They also explored how other parameters, such as feedback consistency, affected the performance of a learning agent. Loftin et al. have further expanded the space of human interaction through detailed investigation of human teaching strategies and developed systems which model the human feedback. Their systems learn in substantially fewer episodes and with less feedback than other approaches [21].